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ABSTRACT 

The advent of Massive Online Open Courses (MOOCs) has 
led to the availability of large educational datasets collected 
from diverse international audiences. Little work has been 
done on the impact of cultural and geographic factors on 
student performance in MOOCs. In this paper, we analyze 
national and cultural differences in students’ performance in 
a large-scale MOOC. We situate our analysis in the context 
of existing theoretical frameworks for cultural analysis. We 
focus on three dimensions of learner behavior: course activ- 
ity profiles; quiz activity profiles; and most connected forum 
peer or best friends. We conclude that countries or associ- 
ated cultural clusters are associated with differences in all 
three dimensions. These findings stress the need for more 
research on the internationalization in online education and 
greater intercultural awareness among MOOC designers. 


1. INTRODUCTION 


Over the past decade there has been a substantial increase in 
the study of cross-cultural behaviors in e-learning systems. 
Prior researchers have shown that learners from different 
cultures behave differently when using educational systems, 
particularly in terms of their off-task behaviors [25, 21], help- 
seeking [21], and collaboration [22, 16]. The cultural differ- 
ences uncovered in these studies suggest that designers of 
future e-learning platforms would benefit from a better un- 
derstanding of their distinct target populations and distinct 
cultures. 


Large-scale MOOCs typically attract diverse international 
audiences. The course we discuss here, for example, at- 
tracted students from 172 countries on 5 continents. Despite 
this acknowledged diversity, most MOOCs take a one-size- 
fits-all approach to designing and structuring the course. 
The materials are typically offered in a single format and 
language, or via direct translations that preserve the struc- 
ture, pacing, and content. 


Prior researchers have shown that country of origin affects 
students’ performance in MOOCs. Nesterko et al. [19] found 
hat non-American students were more prone to complete 
MOOCs and to seek certification than their U.S. counter- 
parts. Guo and Reinecke [12] found that a student’s country 
of origin significantly predicted the amount of content that 
hey would cover and the amount of time that they spent re- 
viewing prior course content. Kizilcec [17] found that there 
was a significant correlation between a country’s level on 
he Human Development Index and the number of students 
from that country who completed a majority of the assign- 
ments. In each of these studies, however, nationality was 
reated as a single independent factor. No substantive com- 
parisons were made between countries or cultures, nor did 
he authors frame their conclusions in the context of prior 
heoretical work on cultural differences in learning. 


A deeper understanding of how students differ both within 
and across cultures will help us to design and deploy more 
effective, and truly international MOOCs. And this under- 
standing will be enriched by relating these differences to 
the rich existing literature on cross-cultural education such 
as Hofstede’s cultural dimensions theory [13] and the Cul- 
tural Dimensions of Learning Framework (CDLF). In this 
paper we will address this need through our analysis of cross- 
cultural student behaviors in an existing MOOC. This was 
an open course with a total enrollment of 29,149 students 
drawn from 172 countries and 5 continents. We found clear 
inter-country and inter-cultural differences in the observed 
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student behaviors and in the distribution of user categories. 
We also found that these differences can be evaluated in the 
context of existing theoretical frameworks and that they are 
consistent with prior educational literature. 


2. LITERATURE REVIEW 
2.1 Culture & Educational Technology 


Advances in educational technology have enabled educators 
to incorporate technologies at larger scale and to collect 
richer and more diverse educational data than ever before. 
This has, in turn, substantially increased interest in study- 
ing variations in the use of e-learning tools across cultures. 


One approach to understanding the impact of culture on 
learning is through field observation. Rodrigo et al. [25] 
coded U.S. and Filipino students’ on- and off-task behaviors 
when using three ITSs. They found that Filipino students 
spent more time on task than their U.S. counterparts on all 
three systems. They also found that the Filipino students 
gamed some systems more than others. Similarly, Ogan et 
al.[22] coded the on-task behaviors and interaction of similar 
students in Chile. They found that the Chilean students had 
a higher proportion of on-task interactions than the U.S. 
students studied previously. 


Another approach is through educational data mining. Ogan 
et al. [21] generated student models from ITS logs collected 
in three countries: Costa Rica, The Philippines, and the 
U.S. Their goal in this work was to predict effective help- 
seeking behaviors. They found that it was possible to gener- 
alize the U.S. model to Filipino students but not to students 
from Costa Rica. Saarela and Karkkainen [27] applied a hi- 
erarchical clustering algorithm to data collected from the 
PISA, a worldwide assessment of 15-year old students cov- 
ering reading, mathematics, and science. They found that 
students’ performance on the test clustered by country, sug- 
gesting cultural influences. 


While these studies found interesting cross-cultural differ- 
ences, we have little understanding of why these differences 
occur, or of how they relate to more general cross-cultural 
variation. Learning behaviors are influenced by a complex 
set of factors and cross-cultural comparisons may help us 
deepen our understanding of this phenomenon and highlight 
ways to remediate or accommodate it. In this paper, we ex- 
plore the logs of student activity in a MOOC, with an eye 
oward how culture may relate to differences in behavior. 


2.2 MOOC Research 


MOOCs represent both opportunities and challenges for ed- 
ucators. On the one hand they involve large numbers of 
users working in highly instrumented systems which can, in 
turn, provide deep insights. On the other hand, however, 
MOOCs have high dropout rates, wide variation in levels of 
engagement, and MOOC users have extremely diverse mo- 
ivations and demographic backgrounds. Thus any insights 
are qualified by the noisy nature of the data. Researchers 
have therefore focused their efforts on better understanding 
MOOC users and their differing behavior patterns. 


One approach to understanding MOOC students is to build 
predictive behavior models based upon their clickstream data, 
such as mining sequences of actions for analysis [29, 5]. 
These induced models are highly accurate but are not always 
readily interpretable. Other work has focused on improving 
our understanding of engagement and dropouts by detecting 
key subgroups. In this work, researchers have used hierar- 
chical clustering to identify groups of students with similar 
patterns of engagement, such as those who viewed many lec- 
tures but rarely attempted quizzes, and those who balanced 
their activities equally [17, 10, 4, 1]. Kizilcec et al. [17] 
and Ferguson et al. [10], for example, clustered students by 
engagement factors such as the number of lectures viewed 
and quizzes attempted. Anderson et al. [1] likewise used 
lecture views and considered the ratio of lectures to assign- 
ments while Bergner et al. [4] focuses solely on assignments 
attempted. These studies served to highlight the distinct 
behavioral patterns of different subgroups. 


Researchers have also begun to study students’ diverse back- 
grounds through voluntary surveys with the goal of under- 
standing how their incoming motivation [28, 2] and demo- 
graphic features [19, 17, 12] affect their observed behaviors. 
Both Nesterko [19] and Deboer [9] found that participation 
(as indicated by survey responses) and certificate attainment 
rates differed across countries, continents, and genders; they 
did not, however, delve deeper into students’ in-system be- 
haviors as logged by the learning environment. Wang and 
Baker [28], by contrast, found that learners receiving course 
certificates tended to be more interested in course content, 
while students not receiving certificates often stated that 
they were seeking a new type of learning experience. 


Few of these researchers however, have focused on the rela- 
tionship between geographic information and observed be- 
haviors. Guo and Reinecke [12] applied linear regression to 
correlate some demographic features such as years of ed- 
ucation to geographic data. They found that a students’ 
country of origin was significantly related to their coverage 
of the course content overall and the extent to which they re- 
viewed prior content, called backjumps. They attributed this 
diversity to varying student-to-teacher ratios. They found 
that countries with a higher ratio had a higher frequency of 
backjumps suggesting more time on review. In related work 
Kizilcec focused on partitioning countries into tiers based 
upon the Human Development Index (HDI). They found 
that as the HDI tier increased, so to did the proportion of 
students who completed the course. While these results are 
instructive, however, the authors made no attempt to situ- 
ate these results in the context of existing theoretical models 
of cross-cultural learning. 


Thus the results from prior MOOC research show that un- 
derstanding students’ diverse backgrounds can be essential 
to the development of effective educational interventions, 
and to providing useful support for student engagement and 
participation. Geographical location, considered as a set of 
economic, cultural, and educational differences, may play a 
crucial role in understanding, supporting, and appealing to 
the increasing population of MOOC users. 
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2.3 Theoretical Frameworks 

MOOCs and educational technologies allow us to collect ro- 
bust information about cross-cultural differences in user be- 
haviors. Yet we face challenges in interpreting and explain- 
ing these results in a consistent theoretical framework. 


Prior educational researchers have worked to identify related 
cultural dimensions and values, and to examine how they 
vary across cultures. One common framework is Hofstede’s 
Cultural Dimensions Theory [13, 14]. Hofstede analyzed a 
set of 117,000 attitude surveys collected by IBM from their 
international workforce and synthesized a set of 7 general 
cultural dimensions: a) power distance; b) collectivism vs. 
individualism; c) femininity vs. masculinity; d) uncertainty 
avoidance; e) long/short term orientation; and f) indulgence 
vs. restraint. Hofstede then calculated scores for each cul- 
ture within these dimensions. 


Hofstede’s dimensions have been used to analyze and ex- 
plain differences in collaboration across cultures [16], as well 
as differences in help-seeking and off-task behavior in ed- 
ucational technology [21, 25]. However these studies have 
suggested that the cultural dimensions framework has some 
limitations in explaining these findings. Many of the key 
differences in the observed behaviors do not correspond to 
the differences that Hofstede’s theory suggests. In particu- 
lar, variations in collectivism and collaboration /help-seeking 
strategies do not seem to relate well to Hofstede’s underly- 
ing dimensions. Therefore we will combine this with the 
Cultural Dimensions Learning Framework (CDLF). 


The CDLF framework, designed by Parrish et al. in 2010 
[24], defines eight cultural parameters regarding social rela- 
tionships, epistemological beliefs, and temporal perceptions, 
and how they manifest in learning situations. The CDLF 
has been used to guide the design and analysis of e-learning 
across cultures [23, 15]. For the purposes of our analysis we 
will focus on the intersection of the CDLF and the Hofst- 
ede dimensions. We will use this hybrid framework to group 
countries into cultural clusters, and to interpret the observed 
behavioral differences between them. Table 3 provides an 
overview of the shared dimensions. 


While these frameworks may help to explain observed be- 
haviors, it is worth noting that learner behaviors in MOOCs 
can be affected by many other factors such as personal mo- 
tivation. Wang and Baker [28], for example, surveyed the 
motivations of incoming students on a later version of the 
course we study here and found that learners who obtained 
course certificates tended to be more interested in course 
content than those who took the MOOC in order to test the 
learning experience. While this highlights the importance 
of individual differences, our analysis below we will focus on 
inter-country differences and cultural factors. 


3. DATA 

The data used in this study was collected from Big Data 
in Education (BDE), an 8-week long MOOC offered by the 
Teacher’s College at Columbia University on the Coursera 
platform [28]. The BDE curriculum included video lectures, 
discussion forums, and 8 weekly assignments or quizzes. The 
lectures covered key methods for educational data analysis. 
The assignments required students to analyze existing data 
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Table 1: Intersection of Hofstede Dimensions and 
the Cultural Dimensions of Learning Framework. 


Hofstede 
[13] 


Dimension 


Selected Interpretations 
in CDLF [24] 


Power Distance: the ex- 
tent to which the less 
powerful members ex- 
pect and accept un- 
equal/unfair situations 


Countries with high power 
distance view teacher as an 
unchallenged authority and 
the primary communicator, 
not as a fallible peer. 


Individualism: the de- 
gree of interdependence 
a society maintains 
among its members 


Highly individualist  stu- 
dents are more prone to 
speak up in class, to value 
diverse opinions in learning, 
and to be motivated by 
personal gain. 


Masculinity: the degree 
to which a culture is mo- 
tivated by competition 
(instead of life quality) 


More masculine cultures are 
associated with increased 
levels of competition and a 
heavier pursuit of recogni- 


tion. 

Students who avoid uncer- 
tainty tend to focus more 
on getting the right answer 
from authoritative sources 
and from the structured 
learning activities. 


Uncertainty Avoidance: 
The extent to which 
a culture feels threat- 
ened by ambiguous or 
unknown situations and 
tries to avoid these 


(typically real data collected from educational settings) and 
to answer questions about their results. All of the assign- 
ments were automatically graded via numeric or multiple- 
choice questions. Students were given between 3 and 5 at- 
tempts to complete each assignment with the best score be- 
ing counted. Students were required to complete their as- 
signments within 2 weeks of it being released. In order to 
obtain a certificate students were required to obtain an aver- 
age grade of > 70% over all 8 assignments. High performing 
students could receive a certificate with distinction. 638 stu- 
dents completed the course and obtained a certificate. 


Data from this course has been previously used to study 
motivation [28], negativity [7], student communities [6], the 
relationship between linguistic quality of forum posts and 
completion[8], as well as longitudinal behavior patterns[31]. 


For the purposes of our analysis we analyzed clickstream 
data containing user IDs, IP addresses, URLs and times- 
tamps for 29,149 students. This data included all 638 stu- 
dents who received a certificate as well as 750 who posted on 
the forum. After classifying students by behavior type we 
found that a total of 1,591 students were actively engaged 
with the course while the remaining 27,588 were ‘bystanders’ 
who enrolled but did not do any significant work. We as- 
signed users to regions based upon their most frequent IP 
address as has been done in prior work [17, 9, 12]. The top 
15 countries by registration are shown in Figure 1. 


We then analyzed the URLs located in the clickstream data 
to identify the following major activities: view lecture (VL), 
attempt or submit quiz (AQ, SQ), and read or make a post 
in forum (RP, MP). We then generated activity sequences 
from this data using an n-gram approach consistent with 
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Figure 1: Number of Registrants from the Top 15 
Countries with Most Registrants 


prior research [29, 5]. Note that this data does not con- 
tain information about how long the student spent view- 
ing a URL. The data only records individual mouseclicks. 
Therefore it functions as a record of student access but not 
a reliable indicator of engagement. 


4. METHODS AND RESULTS 


We hypothesize that students from different countries or cul- 
tures will behave differently in the course. We chose to 
examine four research questions: RQ1. (Course Activity 
Profiles, CAPs) What are the primary categories of stu- 
dents based upon the frequency (both total and relative) 
with which they accessed different course activities? RQ2. 
(CAPs by Country) Does the proportion of student cat- 
egories differ by country? RQ3. (Quiz Activity Profiles, 
QAPs) When do students in each category access the dif- 
ferent types of course activities and how is that correlated 
with quiz submissions? RQ4. (QAPs by Culture & Coun- 
try) How do quiz-based activity profiles and countries relate 
to the four overlapping Hofstede/CDLF cultural dimensions 
of: power distance, individualism, masculinity, and uncer- 
tainty avoidance? RQ5. (Forum best friends) Is a student’s 
most frequent forum partner in the same country/culture? 


For RQ1, we used hierarchical clustering to identify five 
course activity profiles (CAPs) (e.g. students who focused 
solely on quizzes). For RQ2, we clustered countries by the 
proportion of students who fit each CAP in order to de- 
termine whether or not students from a given country are 
more likely to fit one CAP over another. For RQ3, we parti- 
tioned the course data by quizzes and examined whether or 
when students in each CAP accessed the lectures, quizzes, 
and forum content. This led to the development of Quiz 
Activity Profiles (QAPs). For RQ4, we then clustered stu- 
dents based upon their cultural dimensions and compared 
the QAPs by culture and student category (CAP). For RQ5, 
we performed a x” analysis to investigate whether the stu- 
dents’ most frequent interlocutor on the forums were more 
likely to be drawn from the same country/culture. In each 
section below, we will present the methods and results for 
each of these questions in greater detail. 


4.1 RQI1: Course Activity Profiles, CAPs 


What are the primary categories of students based upon the 
frequency (both total and relative) with which they accessed 


different course activities? Prior researchers have used hi- 
erarchical clustering to discover meaningful subgroups such 
as: users who viewed many lectures but rarely attempted 
quizzes and users who balanced the number of lectures viewed 
and quizzes attempted [17, 10, 4, 1]. 


In this work we applied hierarchical clustering to classify stu- 
dents based upon the proportion of activities that they en- 
gaged in over the course. These included: lectures accessed, 
quizzes attempted, and form posts made or accessed. We 
found that clustering students by the the number of lectures 
that they accessed and quizzes attempted yielded five inter- 
pretable clusters which we designated solvers (generally take 
more quizzes), viewers (generally watch more lectures), all- 
rounders (balance both), samplers (watch some lectures and 
do a quiz), and bystanders (do very little). Table 2 shows 
the CAP clusters with average silhouette widths (ASWs) 
in excess of 0.68, which indicates that they are well-chosen 
classifications [26]. These CAPs closely resemble the stu- 
dent types described by Anderson et al. [1] who clustered 
MOOC students based upon the ratio of lectures viewed to 
assignments completed. In this case we used attempts in 
place of submissions. 


Table 2: Course Activity Profile Clusters: size, 
#lectures viewed, #quiz attempts, and _ perfor- 
mance. 


% Certificate 
Distinct} Normal 


CAP Lectures} Quiz 
viewed |Attempts 
(max:54) (max:7) 


Solver = (n=388, IM:5.30  [M:7.67 41.10% | 0.07% 
IASW=0.72): Sd:7.15  |Sd:0.77 

mainly attempt 

quizzes 

[Viewer (n=107, [M:49.57 [M:0.55 0% 0% 


IASW=0.72):mainly |SD:2.95 |SD:0.96 
view lectures 

\A ll-rounder 
(n=519, 
IASW=0.68):balance 
lectures & quizzes 
IBystander 
(n=27558, 
IASW=0.84):do 
little 


IM:45.23 |M:7.58 
Sd:8.3 — {$d:0.89 


79.19% | 8.29% 


IM:1.87 |M:1.25 0% 0% 
Sd:2.72 |Sd:1.43 


As Table 2 shows, the all-rounders have the highest rate of 
certificate completion. For the rest of our analysis we will 
focus on three categories: viewer, solver, and all-rounder. 


4.2 RQ2: CAPs by Country 


Does the proportion of student categories differ by country? 


After identifying the meaningful CAP clusters, we compared 
countries based upon the proportion of CAPs observed. We 
again applied hierarchical clustering on countries with more 
than 15 users from the viewer, solver, and all-rounder stu- 
dents. In this case we found that three clusters yielded the 
highest ASW values. These clusters are shown in Figure 2. 
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Figure 2: Hierarchical clusters of countries by pro- 
portion of user categories. For each country, the 
proportion of user categories is plotted as stacked 
bar, and the sample size is given in parentheses 


This clustering grouped countries with a high proportion 
of solvers in Cluster 1. This includes developing countries, 
Russia, and Singapore. The proportion of solvers present 
in Cluster 1 is significantly higher than that of cluster 3: 
x? (1, N = 740) = 34.95, p < 0.001. 


4.3 RQ3. Quiz Activity Profiles, QAPs 


When do students in each category access the different types 
of course activities and how is that correlated with quiz sub- 
missions ? 


After identifying the CAPs and examining their relative pro- 
portion within countries, we proceeded to analyze the inter- 
country behavioral differences within each CAP. It is our hy- 
pothesis that students from different countries will behave 
differently given the different Hofstede/CDLF dimensions. 
In order to assess this hypothesis we analyzed the behavioral 
differences among users with regards to the course content 
accessed in the three learning phases described below. 


(first quiz (subsequent quiz 


submission) submission) (between quizzes) 


1sttime 1s*time Final quiz 
attempt a quiz submit aquiz submission 


1sttime attempt 
the next quiz 


Figure 3: Illustration of the three learning phases 


In order to better understand when students engaged in 
different learning activities we segmented the activity se- 
quences into three phases based upon the quiz attempts. 
These phases are shown in Figure 3. For each phase we 
counted average number of lectures viewed (VL), forum posts 
made (MP), and posts read (RP). For the first quiz sub- 
mission, and for the subsequent submission phases, we also 
counted the average number of times that a student at- 
tempted and submitted the same quiz (AQ, SQ). We ex- 
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Figure 4: Quiz Activity Profiles for Solvers and All- 
rounders in Three Learning Phases. 


cluded viewers from this analysis as they made little to no 
attempts at the quizzes. 


The relative QAP values for solvers and all-rounders in this 
analysis are shown in Figure 4. We then conducted a series of 
pairwise Kruskal-Wallis tests [20] with Benjamini-Hochberg 
correction [3] comparing the performance by group and learn- 
ing phase to a baseline of the course average. We found that 
the solvers and all-rounders viewed significantly more lec- 
tures between the quizzes and read more posts during sub- 
sequent quiz submissions than in the other learning phases. 


4.4 RQ4. QAPs by Culture 


How do quiz-based activity profiles and countries relate to 
the four overlapping Hofstede/CDLF cultural dimensions of: 
power distance, individualism, masculinity, and uncertainty 
avoidance? 


In order to assess this question we applied hierarchical clus- 
tering on countries with more than 15 all-rounders, solvers 
or viewers, based on the four shared dimensions. This pro- 
duced three clusters with ASWs above 0.46. For our analysis 
we treated the first cluster as the baseline as it contains 
the majority of the student population. Then, for each 
course activity in the learning phases, we conducted a se- 
ries of Kruskal-Wallis tests comparing each QAP by CAP 
and Cluster with the course average baseline. We applied 
Benjamini-Hochberg correction to correct for the multiple 
tests as above. The results are shown in Figure 5. 


Countries in cultural cluster 1 (Australia, Canada, the U.S. 
and U.K. cluster) have the lowest average power distance 
and the highest average individualism. In our analysis we 
found that solvers in clusters 2 (Russia, Spain, Brazil, & 
France) and 3 (China, India, & Singapore) read and made 
more posts during multiple learning phases. These differ- 
ences were significant or marginally-significant. Moreover, 
solvers in cluster 3, whose countries are characterized by 
the highest average power distance and lowest average in- 
dividualism, viewed significantly fewer lectures between the 
quizzes. All-rounders in cluster 3 also viewed significantly 
fewer lectures during the first quiz submission and made 
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Figure 5: Cultural clusters based on Hofstede-CDLF 
values with statistically-significant values (p < 0.05 
bolded) and marginally significant (p < 0.1) QAP 
differences as compared to baseline behaviors. 


more submissions per quiz, this difference was marginally 
significant. 


We found a high degree of overlap between the cultural clus- 
ters and the CAP clusters described in section 4.2. Cul- 
tural cluster 1 is a subset of the all-rounder CAP cluster, 
by country, and cultural cluster 3 is a subset of the solver 
CAP cluster. These results suggest that students from coun- 
tries with higher individualism and lower power distance are 
wice as likely to be all-rounders, while students from coun- 
tries with higher power distance and lower individualism are 
more prone to focus on evaluations. However, cultural clus- 
ter 2 includes students that were evenly split between solvers 
and all-rounders. These findings suggest that the cultural 
dimensions are directly connected to some aspects of the stu- 
dents’ observed behaviors, but other personal motivations 
may also dominate student behaviors. 


4.5 RQ5. Forum “Best Friend” 


Is a student’s most frequent forum partner in the same coun- 
try/culture? 


For this analysis we identified each students’ “best friend” 
based upon their forum interactions. In a prior study, we 
tested whether we can predict students’ performance in the 
course based upon their implicit social relationships in the 
forum [6]. In this case we constructed a similar relationship 
graph for the 750 forum users based upon that work and the 
work of Fire et al. [11]. Edges in the graph were weighted 
based upon the number of times that a user had replied to a 
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thread that the other used had posted in. We then defined 
a students’ “best friend” as the individual with the highest- 
weighted edge between them. 


Then, for each of the top 15 countries and the 3 cultural 
clusters defined in the prior section we performed a x? test 
with the proportion of “best friends” within the cluster as 
the dependent variable. Our goal was to test whether or not 
the cluster was a significant predictor of the proportion of 
individuals with “best friends” in their cluster. The results 
are shown in Table 3. We found that for all three cultural 
clusters, the students are significantly more likely to have a 
best friend within their own country. 


Table 3: Groups whose “best friends” are signifi- 
cantly more likely to be from the same group 


Clusters & | % IN this | % NOT IN | p 
Countries group this group 

with best | with best 

friends in| friends in 

this group | this group 
Cluster1(n=381): | 64.04% 54.09% 0.0065 
Australia, 
Canada,U.S.,U.K. 
Cluster2(n=83): | 36.60% 5.93% <0.001 
Russia, Spain, 
Brazil, France 
Cluster3(n=91): | 19.78% 10.13% 0.0066 
China, India, 
Singapore 
China (n=19) 26.32% 1.99% <0.001 
Brazil (n=38) 63.16% 1.31% <0.001 


5. DISCUSSION 


In this study, we conducted an exploratory analysis on three 
dimensions of MOOC behavior by country and culture. We 
first identified five Course Activity Profiles (CAPs) based 
on the number of lecture views and quiz attempts: viewers, 
solvers, all-rounders, samplers, and bystanders. We found 
that the all-rounder students were most likely to obtain a 
certificate of completion, followed by the solvers. This in- 
dicates that the behavior profiles exhibited by these groups 
are a good indicator of students who are working toward 
certification. 


We then studied the distribution of CAPs over countries. 
To that end we clustered countries with 15 or more stu- 
dents in the solver, viewer, or all-rounder categories based 
upon their CAP distributions. Interestingly we found that 
the developing countries in our dataset all contained a sub- 
stantially higher proportion of solvers than other countries. 
We then clustered the same set of countries using the Hofst- 
ede/CDLF cultural frameworks [13, 24]. We found that the 
resulting cultural clusters also aligned with the observed stu- 
dent types. Our first cultural cluster, which included Aus- 
tralia, Canada, the U.S., and the U.K., was dominated by 
all-rounders while our third cluster, which included China, 
India, and Singapore, was dominated by solvers. This dis- 
tinction may reflect differing educational traditions, as Asian 
countries are historically more test-centric [18, 30]. It may 
also reflect differences in the professional environments of 
the countries as certificates may be more valuable for ca- 
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reer advancement in Asian nations. Indeed, it may be the 
case that the solvers are studying offline and are using the 
MOOC as a certification system. 


Following that we focused on the students’ quiz-centric be- 
havior. We defined the Quiz Activity Profiles (QAPs) based 
upon the students’ major activities between quizzes and be- 
fore subsequent quiz attempts. We found that, regardless of 
the student’s CAP, they typically viewed lectures between 
quizzes, and then turned to forum posts after their initial 
submission and before any resubmission. This resembles 
some traditional classroom settings where students attend 
lectures before doing homework and then only turn to the 
office hours or peers after they face some difficulty. 


When clustering the countries by cultural dimensions we also 
found that two of our clusters were dominated by countries 
with higher power distances and lower individualism (cluster 
2: Russia, Spain, Brazil, and France; cluster 3: China, In- 
dia, & Singapore). Students in these clusters were less likely 
to interact on the forum in most of the learning phases than 
the students in cluster 1 which was dominated by countries 
with low power distance and high individualism. This find- 
ing is consistent with other work on the CDLF which found 
that students in countries with high power distance tend to 
treat the teacher as the unchallenged communicator versus 
students in countries with low power distance who place a 
higher value on dialogue and discussion in the learning pro- 
cess. This framework, however, does not explain the other 
observed variations in cultural cluster 3, notably their ap- 
parent focus on work between quiz attempts. We believe 
hat the explanation may lie in the educational culture of 
his cluster. As noted above this cluster consists entirely of 
Asian nations which are historically test-driven. We believe 
hat this educational culture may cause the students to view 
quizzes as the primary goal, leading them to focus their ef- 
forts on viewing lectures and forums after they have seen 
he quiz. Moreover, this cultural emphasis on exams may 
be the primary reason that Asian students were more prone 
o re-submit quizzes rather than moving on to new material. 


Finally, we analyzed students’ “best friends” on the forums. 
We found that students are more likely to have a “best 
friend” [6, 11] from countries in the same cultural cluster 
as their own. Chinese and Brazilian students, in particular, 
are more likely to have “best friend” from their own country. 
This close connection may be explained by several factors. 
First, students from the same country may have the same 
motivations and overall view of the course which would lead 
them to join forums that fit their shared needs. Second, stu- 
dents may face difficulties in communicating with individu- 
als from other nations due to language barriers, thus making 
them more connected to their neighbors. And third, the ob- 
served relationships in the forums may reflect real offline 
relationships among students who joined the class together 
and are collaborating offline. In the absence of additional 
data we cannot distinguish among these alternatives. 


Ultimately we conclude that students from different coun- 
tries and cultures do exhibit different learner behaviors on 
the BDE MOOC. These differences may be explained by 
country, cultural dimensions, and educational differences. 
We believe that the students’ observed behaviors are driven 
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in part by their own goals and their unique cultural back- 
ground. Students who come from countries that value dis- 
cussion are more prone to interact on the forums. Stu- 
dents who come from countries that are test-centric are more 
prone to focus on improving their quiz scores and will struc- 
ture their efforts around that. These findings contribute 
to our understanding of the role that culture and coun- 
try play in MOOC learner behaviors. They also suggest 
some culturally-influenced behaviors that MOOC designers 
should consider when designing their materials. 


5.1 Conclusions & Future Work 

Our goal in this study was to increase general understand- 
ing of behavioral differences in MOOC populations, and the 
possible role that country and culture may play. We found 
interpretable inter-country and intercultural differences in 
students’ observed activities, both across the whole course 
and when segmented by quizzes. We also found that fo- 
rum users were most strongly connected to individuals from 
their own country or from culturally-related countries. We 
analyzed these findings in the context of a hybrid Hofst- 
ede/CDLF cultural framework and found that our observed 
clusters were consistent with the theoretical literature. 


This paper is one of the first to explore the relationship be- 
tween observed behaviors and learners’ country or culture. 
In future work we plan to examine the generality of these 
findings by analyzing other related MOOCs. Our present 
dataset includes 29,149 accounts identified from the click- 
stream data, only 1,591 of which were non-bystanders, and 
only 750 of whom participated in the forum. While this is 
consistent with other MOOCs, it is also somewhat skewed 
and contains relatively small samples for many countries. 


As we build a better understanding of the interactions be- 
tween culture, behavior, and MOOC performance, new ques- 
tions arise for MOOC designers. Should e-learning platform 
designers intervene to change cultural behaviors? For exam- 
ple, should they encourage students to use forums more or to 
communicate across cultural lines? Or should they consider 
supporting many separate groups by providing language- 
specific forums and tailored tracks? If so, how can we assess 
the impact of such interventions? It may be worthwhile to 
conduct more user-centered research so that we can better 
understand the unique needs of diverse populations. This 
type of work may help us to better understand how to ad- 
dress the diverse needs of such unprecedented student pop- 
ulations. 
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